Determining changes in a driving environment based on vehicle behavior

ABSTRACT

A method and apparatus are provided for determining whether a driving environment has changed relative to previously stored information about the driving environment. The apparatus may include an autonomous driving computer system configured to detect one or more vehicles in the driving environment, and determine corresponding trajectories for those detected vehicles. The autonomous driving computer system may then compare the determined trajectories to an expected trajectory of a hypothetical vehicle in the driving environment. Based on the comparison, the autonomous driving computer system may determine whether the driving environment has changed and/or a probability that the driving environment has changed, relative to the previously stored information about the driving environment.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 17/222,291, filed Apr. 5, 2021, which is a continuation of U.S.patent application Ser. No. 16/225,651, filed Dec. 19, 2018, now U.S.Pat. No. 11,011,061, which is a continuation of U.S. patent applicationSer. No. 15/468,574, filed on Mar. 24, 2017, now U.S. Pat. No.10,192,442, which is a continuation of U.S. patent application Ser. No.13/628,905, filed on Sep. 27, 2012, now U.S. Pat. No. 9,633,564, thedisclosures of which are incorporated herein by reference.

BACKGROUND

An autonomous vehicle may use various computing systems to aid in thetransport of passengers from one location to another. In addition, theautonomous vehicle may require an initial input or continuous input froman operator, such as a pilot, driver, or passenger. Other autonomoussystems, for example autopilot systems, may be used only when the systemhas been engaged, which permits the operator to switch from a manualmode (where the operator exercises a high degree of control over themovement of the autonomous vehicle) to an autonomous mode (where theautonomous vehicle essentially drives itself) to modes that liesomewhere in between.

The autonomous vehicle may be equipped with various types of sensors inorder to detect objects in its environment. For example, the autonomousvehicles may include such sensors as lasers, sonar, radar, cameras, andother sensors that scan and record data from the autonomous vehicle'senvironment. Sensor data from one or more of these sensors may be usedto detect objects and their respective characteristics (position, shape,heading, speed, etc.). This detection and identification is a criticalfunction for the safe operation of the autonomous vehicle.

To navigate an environment confidently and precisely, the autonomousvehicle may rely on a prior stored electronic representation of theenvironment (e.g., a roadway, a highway, etc.). The electronicrepresentation of the environment may be considered a “map” thatidentifies such features as lane markings, lane edges, k-rail concretebarriers, lane dividers, road medians, traffic safety cones, and othersuch features. The autonomous vehicle may store the map for both complexand simple environments.

However, there are times in which these prior stored maps may beout-of-date or inaccurate. For example, there may be construction in theenvironment or an accident occurs on a roadway. As a result, the lanesof the roadway may be shifted relative to their previously indicatedposition in the prior stored map. In such circumstances, the autonomousvehicle must be able to identify when these changes in the roadwayoccur.

BRIEF SUMMARY

An apparatus and method are disclosed. In one embodiment, the apparatusincludes a sensor configured to detect a first vehicle in a drivingenvironment and a computer-readable memory that stores detailed mapinformation for a driving environment, the detailed map informationcomprising information about a road on which the first vehicle travels,and first state information for the first vehicle, the first stateinformation identifying at least one of position, speed, or direction oftravel for the first vehicle. The apparatus may also include a processorin communication with the computer-readable memory and the sensor. Theprocessor may be configured to receive sensor information from thesensor, the sensor information based on having detected the firstvehicle in the driving environment, determine the first stateinformation based on the received sensor information, determine a firsttrajectory based on the first state information, determine an expectedtrajectory based on the detailed map information, and determine that thedriving environment has changed by comparing the determined expectedtrajectory with the determined first trajectory.

In another embodiment of the apparatus, the processor is furtherconfigured to determine a deviation metric value by comparing thedetermined expected trajectory with the determined first trajectory,wherein the processor determines that the driving environment haschanged when the deviation metric value exceeds a deviation metricthreshold.

In a further embodiment of the apparatus, the determined deviationmetric value comprises a maximum deviation metric value representing amaximum difference between the determined first trajectory and thedetermined expected trajectory.

In yet another embodiment of the apparatus, the determined deviationmetric value comprises an average signed deviation metric value, theaverage signed deviation metric value representing a magnitude anddirection of a difference between the determined first trajectory andthe determined expected trajectory.

In yet a further embodiment of the apparatus, the determined firsttrajectory comprises an average trajectory, the average trajectoryhaving been averaged over a predetermined time period.

In another embodiment of the apparatus, the determined expectedtrajectory is based on a centerline of the road corresponding to thedetailed map information.

In a further embodiment of the apparatus, the computer-readable memoryfurther stores a probability model that defines a probability that thedriving environment has changed relative to the detailed map informationbased on at least one deviation metric value determined from thecomparison of the determined first trajectory with the determinedexpected trajectory, and a probability function that determines theprobability that the driving environment has changed relative to thedetailed map information based on the probability model. In addition,the processor may be further configured to determine the probabilitythat the driving environment has changed relative to the detailed mapinformation based on the probability function.

In yet another embodiment of the apparatus, the probability model is oneof a plurality of probability models, and the processor may be furtherconfigured to select the probability model from the plurality ofprobability models based on a first geographic location.

In yet a further embodiment of the apparatus, the determined firsttrajectory comprises a plurality of trajectories, each of thetrajectories of the plurality of trajectories corresponding to a vehiclein the driving environment, and the processor may be further configuredto consolidate the plurality of trajectories as the determined firsttrajectory based on at least one consolidation factor.

In another embodiment of the apparatus, the processor may be furtherconfigured to determine a consolidated trajectory quality value for theplurality of trajectories, the consolidated trajectory quality valuerepresenting a quality of the determined first trajectory, determine theprobability that the driving environment has changed relative to thedetailed map information based on the determined consolidated trajectoryquality value.

In one embodiment of the method, the method may include detecting, witha sensor of an autonomous vehicle, a first vehicle in a drivingenvironment, and receiving, with a processor in communication with thesensor, sensor information based on having detected the first vehicle inthe driving environment. The method may also include determining, withthe processor, the first state information based on the received sensorinformation, the first state information identifying at least one ofposition, speed, or direction of travel for the first vehicle. Themethod may further include determining, with the processor, a firsttrajectory based on the first state information, and determining, withthe processor, an expected trajectory based on detailed map information,the detailed map information comprising information about the drivingenvironment in which the first vehicle travels. The method may alsoinclude determining, with the processor, that the driving environmenthas changed by comparing the determined expected trajectory with thedetermined first trajectory.

In another embodiment of the method, the method may include determining,with the processor, a deviation metric value by comparing the determinedexpected trajectory with the determined first trajectory, anddetermining, with the processor, that the driving environment haschanged when the deviation metric value exceeds a deviation metricthreshold.

In a further embodiment of the method, the determined deviation metricvalue comprises a maximum deviation metric value representing a maximumdifference between the determined first trajectory and the determinedexpected trajectory.

In yet another embodiment of the method, the determined deviation metricvalue comprises an average signed deviation metric value, the averagesigned deviation metric value representing a magnitude and direction ofa difference between the determined first trajectory and the determinedexpected trajectory.

In yet a further embodiment of the method, the determined firsttrajectory comprises an average trajectory, the average trajectoryhaving been averaged over a predetermined time period.

In another embodiment of the method, the determined expected trajectoryis based on a centerline of the road corresponding to the detailed mapinformation.

In a further embodiment of the method, the method includes determining,with the processor, a probability that the driving environment haschanged relative to the detailed map information based on a probabilityfunction, wherein the probability function determines the probabilitythat the driving environment has changed relative to the detailed mapinformation based on a probability model, and the probability modeldefines a probability that the driving environment has changed relativeto the detailed map information based on at least one deviation metricvalue determined from the comparison of the determined first trajectorywith the determined expected trajectory.

In yet another embodiment of the method, the probability model is one ofa plurality of probability models and the method further includesselecting, with the processor, the probability model from the pluralityof probability models based on a first geographic location.

In yet a further embodiment of the method, the determined firsttrajectory comprises a plurality of trajectories, each of thetrajectories of the plurality of trajectories corresponding to a vehiclein the driving environment. The method may also include consolidating,with the processor, the plurality of trajectories as the determinedfirst trajectory based on at least one consolidation factor.

In another embodiment of the method, the method ay include determining,with the processor, a consolidated trajectory quality value for theplurality of trajectories, the consolidated trajectory quality valuerepresenting a quality of the determined first trajectory, anddetermining, with the processor, the probability that the drivingenvironment has changed relative to the detailed map information basedon the determined consolidated trajectory quality value.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are not intended to be drawn to scale. In thedrawings, each identical or nearly identical component that isillustrated in various figures is represented by a like numeral. Forpurposes of clarity, not every component may be labeled in everydrawing. In the drawings:

FIG. 1 illustrates an example of an autonomous vehicle for determiningwhether a driving environment has changed based on tracking one or morevehicles according to aspects of the disclosure.

FIG. 2 illustrates an example of an interior of the autonomous vehicleaccording to aspects of the disclosure

FIG. 3 illustrates an example of the placement of one or more sensors onthe autonomous vehicle according to aspects of the disclosure.

FIGS. 4A-4D illustrate various views of the approximate sensor fields ofthe various sensors on the autonomous vehicle according to aspects ofthe disclosure.

FIG. 5 illustrates an example of detailed map information that may bestored by the autonomous vehicle in accordance with aspects of thedisclosure.

FIG. 6 illustrates an example of the autonomous vehicle detecting one ormore vehicles in the driving environment according to aspects of thedisclosure.

FIG. 7 illustrates an example of the autonomous vehicle determiningtrajectories for detected vehicles in the driving environment accordingto aspects of the disclosure.

FIGS. 8A-8B illustrate examples of using trajectories to determinewhether the driving environment has changed according to aspects of thedisclosure.

FIG. 9 illustrates another example of the autonomous vehicle detectingone or more vehicles in the driving environment according to aspects ofthe disclosure.

FIG. 10 illustrates an example of the autonomous vehicle comparingdetermined trajectories for the detected one or more vehicles from FIG.9 according to aspects of the disclosure.

FIG. 11 illustrates an example of the autonomous vehicle comparingconsolidated trajectories for the detected one or more vehicles fromFIG. 9 according to aspects of the disclosure.

FIG. 12 illustrates a first example of logic flow for determiningwhether the driving environment has changed based on one or moredetected vehicles according to aspects of the disclosure.

FIG. 13 illustrates a second example of logic flow for determiningwhether the driving environment has changed based on one or moredetected vehicles according to aspects of the disclosure

DETAILED DESCRIPTION

This disclosure provides for systems and methods for determining when apreviously stored map is inaccurate. In particular, this disclosureprovides for an autonomous vehicle that evaluates the behavior of one ormore detected vehicles in a driving environment (e.g., a road, ahighway, a parkway, a street, etc.) to determine when, or if, the map ofthe driving environment stored by the autonomous vehicle is accurate.

In one embodiment, the autonomous vehicle may monitor and track thelocation of one or more vehicles in the driving environment anddetermine trajectories for the one or more vehicles. Using a mapdiscrepancy algorithm, the autonomous vehicle may then compare thedetermined vehicle trajectories to expected vehicle trajectories basedon identified lanes in the previously stored map. When the autonomousvehicle observes that one or more vehicles are consistently moving in amanner that does not match the expected behavior based on the previouslystored map, the autonomous vehicle may identify that the map is nolonger reliable (i.e., inaccurate).

For example, in a construction zone, the lanes in the drivingenvironment may be shifted to accommodate the construction work. In thisexample, traffic may be shifted right or left based on newly establishedtemporary lanes and the trajectories for the various vehicles may nolonger follow the previous lanes of the driving environment (i.e., thelanes stored in the map of the autonomous vehicle). When the autonomousvehicle observes and identifies that the traffic has shifted (e.g., bymonitoring a consistent change in vehicle trajectories), the autonomousvehicle may conclude that the previously stored map is inaccurate. Whenthe autonomous vehicle identifies that the previously stored map isinaccurate, the autonomous vehicle may stop relying on its previouslystored map information. Instead, the autonomous vehicle may rely onanother mechanism for maneuvering through the changed drivingenvironment, such as by retrieving a map from a map provider server orrequesting that a passenger in the autonomous vehicle take control.

FIG. 1 illustrates an apparatus 102 for determining whether a drivingenvironment has changed based on the determined trajectories of detectedvehicles. In one embodiment, the apparatus may include an autonomousvehicle 104. The autonomous vehicle 104 may be configured to operateautonomously, e.g., drive without the assistance of a human driver.Moreover, the autonomous vehicle 104 may be configured to detect variousvehicles and determine the trajectories of the detected vehicles whilethe autonomous vehicle 104 is operating autonomously.

While certain aspects of the disclosure are particularly useful inconnection with specific types of vehicles, the autonomous vehicle 104may be any type of vehicle including, but not limited to, cars, trucks,motorcycles, busses, boats, airplanes, helicopters, lawnmowers,recreational vehicles, amusement park vehicles, farm equipment,construction equipment, trams, golf carts, trains, and trolleys.

In one embodiment, the autonomous driving computer system 144 mayinclude a processor 106 and a memory 108. The autonomous drivingcomputer system 144 may also include other components typically presentin a general purpose computer.

The memory 108 may store information accessible by the processor 106,such as instructions 110 and data 112 that may be executed or otherwiseused by the processor 106. The memory 108 may be of any type of memoryoperative to store information accessible by the processor 106,including a computer-readable medium, or other medium that stores datathat may be read with the aid of an electronic device. Examples of thememory 108 include, but are not limited, a hard-drive, a memory card,read-only memory (“ROM”), random-access memory (“RAM”), digital videodisc (“DVD”), or other optical disks, as well as other write-capable andread-only memories. Systems and methods may include differentcombinations of the foregoing, whereby different portions of theinstructions and data are stored on different types of media.

The instructions 110 may be any set of instructions that may be executeddirectly (such as machine code) or indirectly (such as scripts) by theprocessor 106. For example, the instructions 110 may be stored ascomputer code on the computer-readable medium. In that regard, the terms“instructions” and “programs” may be used interchangeably herein. Theinstructions 110 may be stored in object code format for directprocessing by the processor 106, or in any other computer languageincluding scripts or collections of independent source code modules thatare interpreted on demand or compiled in advance. Functions, methods androutines of the instructions 110 are explained in more detail below.

The data 112 may be retrieved, stored, or modified by processor 106 inaccordance with the instructions 110. For instance, although thedisclosed embodiments not limited by any particular data structure, thedata 112 may be stored in computer registers, in a relational databaseas a table having a plurality of different fields and records, XMLdocuments, flat files, or in any computer-readable format. By furtherway of example only, image data may be stored as bitmaps comprised ofgrids of pixels that are stored in accordance with formats that arecompressed or uncompressed, lossless (e.g., BMP) or lossy (e.g., JPEG),and bitmap or vector-based (e.g., SVG), as well as computer instructionsfor drawing graphics. The data 112 may comprise any informationsufficient to identify the relevant information, such as numbers,descriptive text, proprietary codes, references to data stored in otherareas of the same memory or different memories (including other networklocations) or information that is used by a function to calculate therelevant data.

The processor 106 may be any conventional processor, including ReducedInstruction Set Computing (“RISC”) processors, Complex Instruction SetComputing (“CISC”) processors, or combinations of the foregoing.Alternatively, the processor may be a dedicated device such as anapplicant-specific integrated circuit (“ASIC”).

Although FIG. 1 functionally illustrates the processor 106, the memory108, and other elements of the autonomous driving computer system 144 asbeing within the same block, it will be understood by those of ordinaryskill in the art that the processor 106 and the memory 108 may actuallycomprise multiple processors and memories that may or may not be storedwithin the same physical housing. For example, the memory 108 may be ahard drive or other storage media located in a housing different fromthat of the autonomous driving computer system 144.

Accordingly, references to a processor or computer will be understood toinclude references to a collection of processors or computers ormemories that may or may not operate in parallel. Rather than using asingle processor to perform the acts described herein, some of thecomponents, such as steering components and deceleration components, mayeach have their own processor that only performs calculations related tothe component's specific function.

In various embodiments described herein, the processor 106 may belocated remote from the autonomous vehicle 104 and may communicate withthe autonomous vehicle 10 wirelessly. In other aspects, some of theprocesses described herein are executed on a processor disposed withinthe autonomous vehicle 104 and others by a remote processor, includingtaking the acts necessary to execute a single maneuver.

The autonomous driving computer system 144 may include all of thecomponents normally used in connection with a computer, such as acentral processing unit (CPU), a memory (e.g., RAM and internal harddrives) storing data 112 and instructions such as an Internet browser orother software application, an electronic display 122 (e.g., a monitorhaving a screen, a small liquid crystal display (“LCD”) touch-screen orany other electrical device that is operable to display information),one or more user input devices (e.g., a mouse, keyboard, touch screenand/or microphone), as well as various sensors (e.g., a video camera)for gathering the explicit (e.g., a gesture) or implicit (e.g., “theperson is asleep”) information about the states and desires of a person.

The vehicle may also include a geographic position component 136 incommunication with the autonomous driving computer system 144 fordetermining the geographic location of the autonomous vehicle 104. Forexample, the geographic position component 136 may include a GlobalPositioning System (“GPS”) receiver to determine the autonomousvehicle's 104 latitude, longitude and/or altitude position. Otherlocation systems such as laser-based localization systems,inertial-aided GPS, or camera-based localization may also be used toidentify the location of the vehicle. The location of the autonomousvehicle 104 may include an absolute geographical location, such aslatitude, longitude, and altitude as well as relative locationinformation, such as location relative to other vehicles immediatelyaround it, which may be determined with less noise than absolutegeographical location.

The geographic position component 136 may also include other devices incommunication with the autonomous driving computer system 144, such asan accelerometer, gyroscope or another direction/speed detection device138 to determine the direction and speed of the vehicle or changesthereto. By way of example only, the geographic position component 136may determine its pitch, yaw or roll (or changes thereto) relative tothe direction of gravity or a plane perpendicular thereto. Thegeographic position component 136 may also track increases or decreasesin speed and the direction of such changes. The location and orientationdata as set forth herein may be provided automatically to the user, theautonomous driving computer 144, the vehicle central processor 126,other computers and combinations of the foregoing.

The autonomous driving computer system 144 may control the direction andspeed of the autonomous vehicle 104 by controlling various components.By way of example, if the autonomous vehicle 104 is operating in acompletely autonomous mode, the autonomous driving computer system 144may cause the autonomous vehicle 104 to accelerate via the accelerationsystem 130 (e.g., by increasing fuel or other energy provided to theengine), decelerate via the braking system 128 (e.g., by decreasing thefuel supplied to the engine or by applying brakes) and change direction(e.g., by turning the front two wheels). The autonomous driving computersystem 144 may also control one or more systems, such as the signalingsystem 130, when controlling the acceleration system 130 and/or thebraking system 128.

The autonomous driving computer system 144 may also control one or morestatus indicators 118, which may convey the status of the autonomousvehicle 104 and its components to a passenger. For example, theautonomous vehicle 104 may be equipped with an electronic display 122for displaying information relating to the overall status of thevehicle, particular sensors, or information about or from the autonomousdriving computer system 144. The electronic display 122 may displaycomputer-generated images of the vehicle's surroundings including, forexample, the status of the autonomous driving computer system 144, theautonomous vehicle 104 itself, roadways, intersections, as well as otherobjects and information.

The autonomous driving computer system 144 may use visual or audiblecues to indicate whether it is obtaining valid data from one or moresensors, whether the it is partially or completely controlling thedirection or speed of the autonomous vehicle 104 or both, such aswhether there are any errors, etc. In addition, the autonomous drivingcomputer system 144 may also have external indicators which indicatewhether, at the moment, a human or an automated system is in control ofthe vehicle, that are readable by humans, other computers, or both.

The autonomous driving computer system 144 may also communicate withother components of the autonomous vehicle 104. For example, autonomousdriving computer system 144 may communicate with a vehicle centralprocessor 126. The autonomous driving computer system 144 may also sendand receive information from the various systems of the autonomousvehicle 104. Communicating with the various systems may includecommunicating with the braking system 128, the acceleration system 130,the signaling system 132, and the vehicle navigation system 134.Communications with these systems may facilitate the control of themovement, speed, etc. of the autonomous vehicle 104. In addition, whenengaged, autonomous driving computer system 144 may control some or allof these functions of the autonomous vehicle 104 and thus be fully orpartially autonomous. It will be understood that although varioussystems and the autonomous driving computer system 144 are shown withinthe autonomous vehicle 104, these systems and components may be externalto the autonomous vehicle 104 or physically separated by largedistances.

The autonomous vehicle 104 may include components for detecting objectsexternal to it, such as other vehicles, obstacles in the roadway,traffic signals, signs, trees, etc. The detection system may includelasers, sonar, radar, cameras or any other detection devices. Forexample, where the autonomous vehicle 104 is a small passenger car, thesmall passenger car may include a laser mounted on the roof or otherconvenient location. In one aspect, the laser may measure the distancebetween the autonomous vehicle 104 and the object surfaces facing theautonomous vehicle 104 by spinning on its axis and changing its pitch.The autonomous vehicle 104 may also include various radar detectionunits, such as those used for adaptive cruise control systems. The radardetection units may be located on the front and back of the car as wellas on either side of the front bumper. In another example, a variety ofcameras may be mounted on the autonomous vehicle 104 at known distancesfrom one another. In this manner, the parallax from the different imagesmay be used to compute the distance to various objects captured by theone or more cameras. These sensors may assist the vehicle in respondingto its environment to maximize safety for passengers as well as objectsor people in the environment.

In addition to the sensors described above, the autonomous drivingcomputer system 144 may also use input from sensors found innon-autonomous vehicles. As examples, these sensors may include tirepressure sensors, engine temperature sensors, brake heat sensors, breakpad status, tire tread sensors, fuel sensors, oil level and qualitysensors, air quality sensors (for detecting temperature, humidity, orparticulates in the air), etc.

The data provided by these sensors may be processed by the autonomousdriving computer system 144 in real-time. In this context, the sensorsmay continuously update their output to reflect the environment beingsensed at or over a range of time, and continuously or as demanded. Thesensors may provide the updated output to the autonomous drivingcomputer system 144 so that it can determine whether the autonomousvehicle's 104 then-current direction or speed should be modified inresponse to the sensed environment.

The autonomous vehicle 104 may also include persistent data fordetecting vehicles and determining the trajectories of the detectedvehicles using one or more of the sensors described above. For example,the data 112 may include detailed map information 114 that defines oneor more driving environments. The detailed map information 114 mayinclude various maps that identify the shape and elevation of roadways,lane lines, intersections, crosswalks, speed limits, traffic signals,buildings, signs, real time traffic information, or other such objectsand information. The detailed map information 114 may further includeexplicit speed limit information associated with various roadwaysegments. The speed limit data may be entered manually or scanned frompreviously taken images of a speed limit sign using, for example,optical-character recognition. In addition, the detailed map information114 may include three-dimensional terrain maps incorporating one or moreof the objects (e.g., crosswalks, intersections, lane lines, etc.)listed above.

The detailed map information 136 may also include lane markerinformation identifying the location, elevation, and shape of lanemarkers. The lane markers may include features such as solid or brokendouble or single lane lines, solid or broken lane lines, reflectors,etc. A given lane may be associated with left and right lane lines orother lane markers that define the boundary of the lane. Thus, mostlanes may be bounded by a left edge of one lane line and a right edge ofanother lane line.

To determine the trajectories of detected vehicles, the autonomousdriving computer system 144 may monitor vehicles in a drivingenvironment corresponding to the detailed map information 114. Forexample, the autonomous driving computer system 144 may detect and trackvehicles at an intersection, on the various types of roadways, and inother such driving environments. As another example, the autonomousdriving computer system 144 may detect and track vehicles enteringand/or exiting a highway, such as vehicles entering the highway via anon-ramp, exiting the highway via an off-ramp and other such behaviors.The autonomous driving computer system 144 may collect, process, andstore this information as part of the vehicle data 116.

In addition, the autonomous driving computer system 144 may refer to thedetailed map information 114 to determine the various vehicletrajectories. More specifically, the autonomous driving computer system144 may cross-reference the position of a detected vehicle with alocation in the detailed map information 114. Based on thiscross-reference, the autonomous driving computer system 144 may thendetermine the trajectory that a detected vehicle takes based on itsdetected position. For example, where the autonomous driving computersystem 144 detects vehicle on a highway, such as a motorcycle, lighttrunk, or other vehicle, the autonomous driving computer system 144 maycross-reference the position of the detected vehicle and determine thetrajectory of the detected vehicle, such as by tracking the positions ofthe vehicle as it continues along the highway. The positions recorded bythe autonomous driving computer system 144 may be stored as part of thevehicle data 116 and may be used in determining the trajectory of thedetected vehicle.

In monitoring vehicles in various driving environments, the data 112 mayinclude vehicle data 116 that defines one or more parameters forclassifying a vehicle. Classifications of vehicle may include suchclassifications as “passenger car,” “bicycle,” “motorcycle,” and othersuch classifications. The parameters defined by the vehicle data 116 mayinform the autonomous driving computer system 144 as to the type ofvehicle detected by a given sensor. For example, the vehicle data 116may include parameters that define the type of vehicle when the vehicleis detected by one or more of the camera sensors, one or more of thelaser sensors, and so forth.

Vehicles may be identified through a vehicle detection algorithm 124,which the processor 106 may use to classify vehicles based on variouscharacteristics, such as the size of the vehicle (bicycles are largerthan a breadbox and smaller than a car), the speed of the vehicle(bicycles do not tend to go faster than 40 miles per hour or slower than0.1 miles per hour), and other such characteristics. In addition, thevehicle may be classified based on specific attributes of the vehicle,such as information contained on a license plate, bumper sticker, orlogos that appear on the vehicle.

The vehicle data 116 may also include state, positional, and/ortrajectory information collected by the autonomous driving computersystem 144 when a vehicle is detected. The autonomous driving computersystem 144 may collect the state and/or positional information about adetected vehicle to assist in the determination of the vehicletrajectory. The vehicle trajectory for the detected vehicle may definethe direction and speed that a vehicle has when in a given drivingenvironment. The vehicle trajectory may also define the past positions,directions, and speed that the detected vehicle had while in the drivingenvironment. As discussed below, the vehicle trajectory for a detectedvehicle may be derived from state information that the autonomousdriving computer system 144 records.

State information may include characteristics about the detectedvehicle. Examples of state information include, but are not limited to,the detected vehicle's speed, the path traveled by the vehicle, thedriving environment in which the vehicle is traveling, any directionalor orientation changes by the vehicle, or other such state information.The state information may also be associated with one or more segmentsof the detailed map information 114 to further refine the state of thedetected vehicle. For example, where the detected vehicle is detected asbeing on a highway (as defined by the detailed map information 114), thecollected state information may identify that the detected vehicle wastraveling in a highway, and may further identify the direction of thedetected vehicle, various positional information or changes about thedetected vehicle (e.g., the original starting lane of the detectedvehicle, the ending lane of the detected vehicle), and other such stateinformation.

The state information collected by the autonomous driving computersystem 144 may be used to determine a trajectory for the detectedvehicle. As discussed previously, based on the detected positions andspeed of the detected vehicle, the autonomous driving computer system144 may derive a trajectory for the detected vehicle. As discussedbelow, this determined vehicle trajectory may be compared against anexpected vehicle trajectory (which may be stored as part of, or derivedfrom, the detailed map information 114) to determine whether the drivingenvironment has changed. In one embodiment, the instructions 110 mayinclude a map discrepancy algorithm 146 that facilitates thedetermination by the autonomous driving computer system 144 as towhether the driving environment has changed.

The autonomous vehicle 104 may collect state information about detectedvehicles regardless of whether the autonomous vehicle 104 is operatingin an autonomous mode or a non-autonomous mode. Thus, whether theautonomous vehicle 104 is operating by itself or has a driver, theautonomous vehicle 104 may collect state and object information todetermine the aforementioned vehicle trajectories.

The autonomous vehicle 104 may also communicate with a map providerserver 142 via a network 140. The network 140 may be implemented as anycombination or type of networks. As examples, the network 140 may be oneor more of a Wide Area Network (“WAN”), such as the Internet; a LocalArea Network (“LAN”); a Personal Area Network (“PAN”), or a combinationof WANs, LANs, and PANs. Moreover, the network 140 may involve the useof one or more wired protocols, such as the Simple Object AccessProtocol (“SOAP”); wireless protocols, such as 802.11a/b/g/n, Bluetooth,or WiMAX; transport protocols, such as TCP or UDP; an Internet layerprotocol, such as IP; application-level protocols, such as HTTP, acombination of any of the aforementioned protocols, or any other type ofprotocol.

The autonomous vehicle 104 may communicate with the map provider server142 to obtain a map of a driving environment. For example, theautonomous vehicle 104 may request a map of the driving environment fromthe map provider server 142 when the autonomous vehicle 104 determinesthat the previously stored map (e.g., the detailed map information 114)is inaccurate. The map received from the map provider server 142 may bean up-to-date map of the driving environment that includes the changesto the driving environment detected by the autonomous vehicle 104. Inaddition, the received map may replace a portion of the detailed mapinformation 114, such as a predetermined geographic area near orsurrounding the driving environment (e.g., a two square mile region orother similar area). By requesting a replacement portion of the detailedmap information 114 from the map provider server 142, rather than acomplete replacement of the detailed map information 114, the autonomousvehicle 114 saves network bandwidth and transfer time.

In one embodiment, the map provider server 142 may comprise a pluralityof computers, e.g., a load balancing server farm, that exchangeinformation with different nodes of a network for the purpose ofreceiving, processing and transmitting data, including the detailed mapinformation 114 and/or vehicle data 116, from the autonomous drivingcomputer system 144. The map provider server 142 may be configuredsimilarly to the autonomous driving computer system 144 (i.e., having aprocessor 106, a memory 108, instructions 110, and data 112).

FIG. 2 illustrates an example of an interior of the autonomous vehicle104 according to aspects of the disclosure. The autonomous vehicle 104may include all of the features of a non-autonomous vehicle, forexample: a steering apparatus, such as steering wheel 210; a navigationdisplay apparatus, such as navigation display 215; and a gear selectorapparatus, such as gear shifter 220. The vehicle 104 may also havevarious user input devices, such as gear shifter 220, touch screen 217,or button inputs 219, for activating or deactivating one or moreautonomous driving modes and for enabling a driver or passenger 290 toprovide information, such as a navigation destination, to the autonomousdriving computer 106.

The autonomous vehicle 104 may also include one or more additionaldisplays. For example, the autonomous vehicle 104 may include a display225 for displaying information regarding the status of the autonomousvehicle 104 or its computer. In another example, the autonomous vehicle104 may include a status indicating apparatus such as status bar 230, toindicate the current status of vehicle 104. In the example of FIG. 2 ,the status bar 230 displays “D” and “2 mph” indicating that theautonomous vehicle 104 is presently in drive mode and is moving at 2miles per hour. In that regard, the autonomous vehicle 104 may displaytext on an electronic display, illuminate portions of the autonomousvehicle 104, such as the steering wheel 210, or provide various othertypes of indications.

FIG. 3 illustrates one example of the autonomous vehicle 104 and theplacement of its one or more sensors. The autonomous vehicle 104 mayinclude lasers 302 and 304, for example, mounted on the front and top ofthe autonomous vehicle 104, respectively. The laser 302 may have a rangeof approximately 150 meters, a thirty-degree vertical field of view, andapproximately a thirty-degree horizontal field of view. The laser 304may have a range of approximately 50-80 meters, a thirty-degree verticalfield of view, and a 360-degree horizontal field of view. The lasers302-304 may provide the autonomous vehicle 104 with range and intensityinformation that the processor 106 may use to identify the location anddistance of various objects. In one aspect, the lasers 302-304 maymeasure the distance between the vehicle and object surfaces facing thevehicle by spinning on its axes and changing their pitch.

The autonomous vehicle 104 may also include various radar detectionunits, such as those used for adaptive cruise control systems. The radardetection units may be located on the front and back of the car as wellas on either side of the front bumper. As shown in the example of FIG. 3, the autonomous vehicle 104 includes radar detection units 306-312located on the side (only one side being shown), front and rear of thevehicle. Each of these radar detection units 306-312 may have a range ofapproximately 200 meters for an approximately 18 degree field of view aswell as a range of approximately 60 meters for an approximately 56degree field of view.

In another example, a variety of cameras may be mounted on theautonomous vehicle 104. The cameras may be mounted at predetermineddistances so that the parallax from the images of two or more camerasmay be used to compute the distance to various objects. As shown in FIG.3 , the autonomous vehicle 104 may include two cameras 314-316 mountedunder a windshield 318 near the rear view mirror (not shown).

The camera 314 may include a range of approximately 200 meters and anapproximately 30 degree horizontal field of view, while the camera 316may include a range of approximately 100 meters and an approximately 60degree horizontal field of view.

Each sensor may be associated with a particular sensor field in whichthe sensor may be used to detect objects. FIG. 4A is a top-down view ofthe approximate sensor fields of the various sensors. FIG. 4B depictsthe approximate sensor fields 402 and 404 for the lasers 302 and 304,respectively based on the fields of view for these sensors. For example,the sensor field 402 includes an approximately 30 degree horizontalfield of view for approximately 150 meters, and the sensor field 404includes a 360-degree horizontal field of view for approximately 80meters.

FIG. 4C depicts the approximate sensor fields 406-420 and for radardetection units 306-312, respectively, based on the fields of view forthese sensors. For example, the radar detection unit 306 includes sensorfields 406 and 408. The sensor field 406 includes an approximately 18degree horizontal field of view for approximately 200 meters, and thesensor field 408 includes an approximately 56 degree horizontal field ofview for approximately 80 meters.

Similarly, the radar detection units 308-312 include the sensor fields410/414/418 and sensor fields 412/416/420. The sensor fields 410/414/418include an approximately 18 degree horizontal field of view forapproximately 200 meters, and the sensor fields 412/416/420 include anapproximately 56 degree horizontal field of view for approximately 80meters. The sensor fields 410 and 414 extend passed the edge of FIGS. 4Aand 4C.

FIG. 4D depicts the approximate sensor fields 422-424 of cameras314-316, respectively, based on the fields of view for these sensors.For example, the sensor field 422 of the camera 314 includes a field ofview of approximately 30 degrees for approximately 200 meters, andsensor field 424 of the camera 316 includes a field of view ofapproximately 60 degrees for approximately 100 meters.

In general, an autonomous vehicle 104 may include sonar devices, stereocameras, a localization camera, a laser, and a radar detection unit eachwith different fields of view. The sonar may have a horizontal field ofview of approximately 60 degrees for a maximum distance of approximately6 meters. The stereo cameras may have an overlapping region with ahorizontal field of view of approximately 50 degrees, a vertical fieldof view of approximately 10 degrees, and a maximum distance ofapproximately 30 meters. The localization camera may have a horizontalfield of view of approximately 75 degrees, a vertical field of view ofapproximately 90 degrees and a maximum distance of approximately 10meters. The laser may have a horizontal field of view of approximately360 degrees, a vertical field of view of approximately 30 degrees, and amaximum distance of 100 meters. The radar may have a horizontal field ofview of 60 degrees for the near beam, 30 degrees for the far beam, and amaximum distance of 200 meters. Hence, the autonomous vehicle 104 may beconfigured with any arrangement of sensors, and each of these sensorsmay capture one or more raw images for use by the object detector 130 todetect the various objects near and around the autonomous vehicle 104.

FIG. 5 illustrates an example of a portion of a detailed map 502 thatmay represent the driving environment of the autonomous vehicle 104. Thedetailed map 502 may be retrieved or referenced by the autonomousvehicle 104 based on a detected position of the autonomous vehicle 104.The detailed map 502 may be stored as part of the detailed mapinformation 114.

The detailed map 502 may further represent a section of a road, such ashighway, parkway, etc., and may include lane information such asinformation about a solid lane line 504, broken lane lines 506, 508, anddouble solid lane lines 510. These lane lines may define lanes 512 and514. Each lane may be associated with a centerline rail 516, 518 whichmay indicate the direction in which a vehicle should generally travel inthe respective lane. For example, a vehicle may follow centerline rail518 when driving along lane 514. In this example, the lane 512 may bebounded by a right lane line 504 and a left lane line 506, and the lane514 is bounded by a right lane line 506 and a left lane line 510. Theedges for lane 512 are edges 520, 522 while the edges for lane 514 areedges 524, 526.

In the example shown in FIG. 5 , the detailed map information 114 may bedepicted as an image-based map. However, the detailed map information114 need not be entirely or completely image-based (e.g., raster-based).For example, the detailed map information 114 may include one or moreroadgraphs or graph networks of information such as roads, lanes,intersections, and the connections between these features. Each featuremay be stored as graph data and may be associated with information suchas a geographic location and whether or not it is linked to otherrelated features, for example, a stop sign may be linked to a road andan intersection, etc. In some examples, the associated data may includegrid-based indices of a roadgraph to allow for efficient lookup ofcertain roadgraph features.

The detailed map information 114 may be loaded into the memory 108 ofthe autonomous vehicle 104 at a predetermined time. In one embodiment,the detailed map information 114 may be loaded into the memory 108 ofthe autonomous vehicle 104 on a daily basis. Alternatively, or inaddition, the detailed map information 114 may be loaded into the memory108 at other predetermined times, such as on a monthly or weekly basis.

In addition, and as discussed previously, the detailed map information114 may be transferred, or received from, the map provider server 142.Receiving detailed map information 114 from the map provider sever 142may also include receiving updated detailed map information, includingany changes to the driving environment that have occurred since thedetailed map information 114 was last transferred to the autonomousdriving computer system 144. In one embodiment, the detailed mapinformation 114 may be transferred, or received from, the map providerserver 142 when the autonomous driving computer system 144 detects achange in the driving environment corresponding to the detailed mapinformation 114.

Referring back to FIG. 1 , the instructions 110 may include variousalgorithms for determining whether the driving environment has changedrelative to its corresponding representation in the detailed mapinformation 114. As discussed previously, the autonomous drivingcomputer system 144 may leverage the vehicle detection algorithm 124 andthe map discrepancy algorithm 146 for determining whether such changeshave occurred.

The vehicle detection algorithm 124 may facilitate the detection ofvehicles by the autonomous driving computer system 144. FIG. 6illustrates an example 602 of the autonomous vehicle 104 in a drivingenvironment. In the example 602, the detected vehicles 604-610 and theautonomous vehicle 104 are driving in a driving environmentcorresponding to the detailed map 502 of FIG. 5 . For contrast, thedetailed map 502 is shown in dotted lines while the autonomous vehicle104 and the detected vehicles 604-610 are shown with solid lines.Moreover, the example 602 illustrates that a change in the drivingenvironment has occurred (i.e., the lanes having been shifted to theright) since the detailed map 502 was provided to the autonomous vehicle104. The shift in the lanes is evidenced by the fact that the vehicles604-610 in proximity to the autonomous vehicle 104 are travellingoff-center relative to the lanes of the detailed map 502.

The autonomous driving computer system 144 may detect and track theseveral vehicles 604-610 in the driving environment based on the vehicledetection algorithm 124 and the various sensors 302-316 mounted to theautonomous vehicle 104. In the example 602, each of the vehicles 604-610are traveling in the same direction as the autonomous vehicle 104, buttheir distances to the autonomous vehicle 104 may vary.

As examples, the autonomous driving computer system 144 may detect afirst vehicle 604 using one or more of the cameras 314-316, a secondvehicle 606 using the radar detection unit 310 and/or the laser sensor304, a third vehicle 608 using the radar detection unit 310 and/or thelaser sensor 304, and a fourth vehicle 610 using the radar detectionunit 312 and/or the laser sensor 304. Other sensors may also be used todetect the vehicles 604-610, such as the laser sensor 302 being used todetect the first vehicle 604. As discussed previously, the autonomousdriving computer system 144 may track the vehicles 604-610 by storingvehicle type and state information, such as position, direction, speed,and other such state information, about the detected vehicles 604-610 inthe memory 108 as part of the vehicle data 116.

Moreover, while not shown in FIG. 6 , the autonomous driving computersystem 144 may also detect and track vehicles traveling in an oppositedirection. For example, the autonomous driving computer system 144 maydetect and track vehicles traveling in an oncoming direction to theautonomous vehicle 104, such as by using one or more of the laser sensor304, the laser sensor 302, the radar detection unit 308, the camerasensors 314-316, or any other combination of sensors mounted to theautonomous vehicle 104.

In tracking the vehicles 604-610, the autonomous driving computer system144 may track one or more of the vehicles 604-610 for a predeterminedperiod of time, until the occurrence of a predetermined condition, or acombination of the two. In tracking the vehicles 604-610 for apredetermined period of time, the autonomous driving computer system 144may track the vehicles for any period of time including, but not limitedto, seconds, minutes, hours, or any other period of time. Examples ofpredetermined conditions include loss of sight of a tracked vehicle, thetracked vehicle moving a predetermined distance away from the autonomousvehicle 104 (e.g., moving out of range of one or more of the sensors),the autonomous driving computer system 144 determining that it hascollected sufficient state information about the tracked vehicle, orother such predetermined conditions. In determining whether sufficientstate information has been collected for a tracked vehicle, theautonomous driving computer system 144 may determine whether it hascollected enough state information to determine a trajectory for thetracked vehicle.

After tracking one or more of the vehicles 604-610, the autonomousdriving computer system 144 may determine corresponding trajectories forthe tracked vehicles 604-610. FIG. 7 illustrates an example 702 of theautonomous driving computer system 144 having determined trajectories704-710 for the tracked vehicles 604-610. As shown in FIG. 7 , theautonomous driving computer system 144 may have determined a firsttrajectory 704 corresponding to the first vehicle 604, a secondtrajectory 706 corresponding to the second vehicle 606, a thirdtrajectory 708 corresponding to the third vehicle 608, and a fourthtrajectory 710 corresponding to the fourth vehicle 610. In general, thetrajectories 704-710 may represent the paths the corresponding vehicles604-610 travelled in the driving environment and their location relativeto the detailed map information 114.

The trajectories 704-710 may include slight lateral shifts as it isgenerally understood that vehicles do not typically travel in absolutelystraight lines. These slight lateral shifts are generally due tocorrections by a human driver (e.g., a driver turning a steering wheelslightly to the right or left to keep a vehicle driving in a relativelystraight path). Each of the trajectories 704-710 may be determined basedon the collected state information for the corresponding vehicles604-610. In one embodiment, a trajectory may include positionalinformation for a corresponding vehicle (e.g., a series of latitudinaland longitudinal points), where the positional information is associatedwith one or more timestamps (e.g., relative timestamps or absolutetimestamps). Further still, a trajectory may be derived as aninterpolated series of positions based on the detected positions of thecorresponding vehicle.

In addition, the trajectories 704-710 may be determined based on one ormore conditions, such as distance, time, or other such factors. Forexample, the autonomous driving computer system 144 may determine thetrajectories 704-710 based on having sufficient distance information fora corresponding vehicle (e.g., 100 feet, 300 meters, two miles, or otherdistance information). As another example, the autonomous drivingcomputer system 144 may determine the trajectories 704-710 aftermonitoring the corresponding vehicles for a sufficient amount of time(e.g., two minutes, ten minutes, an hour, etc.). The autonomous drivingcomputer system 144 may also determine the trajectories 704-710 based oncombinations and/or variations of these conditions.

Returning to the example of FIG. 7 , the autonomous driving computersystem 144 may then use the trajectories 704-710 to determine whetherchanges in the driving environment have occurred. For example, theautonomous driving computer system 144 may determine individually orconsolidated average trajectories from the trajectories 704-710, comparethe averaged trajectories with the detailed map information 144, andthen determine whether there is a consistent bias in the averagedtrajectories. The consistent bias may indicate that a change (e.g., ashift in one or more lanes) in the detailed map information 144 hasoccurred.

In one embodiment, the autonomous driving computer system 144 may usethe individual trajectories 704-710 to determine whether any changeshave occurred. FIGS. 8A and 8B illustrate examples 802-804 of theautonomous driving computer system 144 using the individual trajectories704-710 to determine whether the driving environment has changed. In theexamples of 802-804, the autonomous vehicle computer system 144 hasindividually averaged the determined trajectories 704-710 to producecorresponding smoothed trajectories 806-812. The smoothed trajectoriesmay lack some of the lateral shifts (e.g., oscillations) that arepresent in the determined trajectories 704-710. The smoothedtrajectories 806-812 may be used to determine whether there is aconsistent bias in the trajectories of the corresponding vehicles604-610 relative to the detailed map information 114. While theautonomous driving computer system 144 may use the smoothed trajectories806-812 to determine whether the detailed map information 114 isinaccurate, it is also possible that the autonomous driving computersystem 114 may use the unsmoothed (i.e., raw) trajectories 704-710 tomake this determination.

To determine whether the detailed map information 114 is inaccurate, theautonomous driving computer system 144 may identify one or more expectedtrajectories for a hypothetical vehicle. In one embodiment, an expectedtrajectory may be based on the centerline for a road lane selected fromthe detailed map information 114. For example, and with reference toFIG. 5 , an expected trajectory, such as the trajectory 814 of FIG. 8A,for lane 514 may be based on the centerline rail 518, and an expectedtrajectory, such as the trajectory 816 of FIG. 8B, for lane 512 may bebased on the centerline rail 516. In another embodiment, the expectedtrajectory may be based on empirical data from prior observations andtracking of one or more vehicles in the driving environment.

The smoothed trajectories 806-808 may then be compared with the one ormore expected trajectories 814-816. The autonomous driving computersystem 144 may determine which expected trajectory to compare with thesmoothed trajectories 806-808 by comparing the positional information ofthe corresponding vehicles 604,610 with the positional information ofthe autonomous vehicle 104. As the autonomous driving computer system144 may have a presumption about which lane the autonomous vehicle 104is traveling (even if that presumption is inaccurate), the result ofcomparing the positional information of the vehicles 604,610 with thepositional information of the autonomous vehicle 104 may indicatewhether the vehicles are traveling within the same lane of theautonomous vehicle 104 or a different lane.

In comparing the smoothed trajectories with the one or more expectedtrajectories, the autonomous driving computer system 144 may determineone or more sets of deviation values. These deviation values may bedetermined by comparing one or more of the expected trajectories withone or more of the determined trajectories (i.e., the smoothed and/orunsmoothed trajectories).

For example, as shown in FIG. 8A, a first set of deviation values 818may correspond to the difference in distance between one or morepositions along the smoothed trajectory 806 (or unsmoothed/rawtrajectory 706) and corresponding one or more positions along theexpected trajectory 814. A second set of deviation values 820 maycorrespond to the difference in distance between one or more positionsalong the smoothed trajectory 808 and corresponding one or morepositions along the expected trajectory 814. The first set of deviationvalues 818 and/or the second set of deviation values 820 may be measuredby any distance type, such as millimeters, centimeters, feet, yards,inches, and so forth.

In the second example 804, the autonomous driving computer system 144may obtain deviation values 822-824 by comparing smoothed trajectories810,812 with a second expected trajectory 816. A third set of deviationvalues 822 may correspond to the difference in distance between one ormore positions along the smoothed trajectory 810 and corresponding oneor more positions along the expected trajectory 816, and a fourth set ofdeviation values 824 may correspond to the difference in distancebetween one or more positions along the smoothed trajectory 812 andcorresponding one or more positions along the expected trajectory 816.

Although FIGS. 8A-8B illustrate examples involving two trajectories each(here, trajectories 806-808 and trajectories 810-812), the autonomousdriving computer system 144 may determine more or fewer trajectories forcomparing with an expected trajectory. For example, the autonomousdriving computer system 144 may be configured to determine a thresholdnumber of trajectories for comparing with an expected trajectory. Thethreshold number of trajectories may be any number of trajectories,e.g., one, two, ten, or any other number. In addition, the number oftrajectories may include any combination of unique trajectories, whereeach trajectory corresponds to a unique vehicle, or non-uniquetrajectories, where different trajectories may correspond to the samevehicle.

In comparing the determined trajectories with the expected trajectories,the autonomous driving computer system 144 may obtain a variety ofmetrics for determining whether the driving environment has changed, andthus, whether the detailed map information 114 is inaccurate. Thesemetrics may include, for example, a maximum deviation metric, an averagedeviation metric, and an average signed deviation metric.

The maximum deviation metric may indicate the maximum difference betweena determined trajectory and an expected trajectory. With reference tothe examples 802-804, the second set of deviation values 820 and thefourth set of deviation values 824 may each include a maximum deviationvalue for each set of compared trajectories (i.e., trajectories 806-808and trajectories 810-812).

The average deviation metric may indicate the average difference betweenthe determined trajectory and the expected trajectory. The averagedeviation for example 802 may be determined by averaging, individuallyor combined, the first set of deviation values 818 and the second set ofdeviation values 820. Similarly, the average deviation for the example804 may be determined by averaging, individually or combined, the thirdset of deviation values 822 and the fourth set of deviation values 824.

The average signed deviation metric may indicate whether the differencebetween the determined trajectory and the expected trajectory isnegative (i.e., the trajectories have been laterally shifted to theleft), positive (i.e., the trajectories have been laterally shifted tothe right), or neutral (i.e., the trajectories have not been shifted).As with the average deviation, the average signed deviation for theexample 802 may be determined based on determining a signed average,individually or combined, of the first set of deviation values 818 andthe second set of deviation values 820. A similar determination may alsobe made for the example 804.

One or more of the foregoing metrics may be used to determine whether,and to what extent, a change has occurred in the driving environment.For example, the maximum deviation metric and/or the average deviationmetric(s) may be compared with a threshold deviation metric to indicatewhether a changed has, in fact, occurred. Where the maximum deviationmetric and/or the average deviation metric exceeds the thresholddeviation metric, the autonomous vehicle computer system 144 maydetermine that there has been a change in the driving environment.Moreover, the average signed deviation metric(s) may indicate the degreeand direction of the change in the driving environment.

For example, with reference to FIG. 8A, the average signed deviationmetric(s) may be positive, since the smoothed trajectories 806-808 areillustrated to the right of the expected trajectory 814. Similarly, theaverage signed deviation metric(s) for FIG. 8B may be positive since thesmoothed trajectories 810-812 are also to the right of the expectedtrajectory 816. Further still, the magnitude of the average signeddeviation metric(s) may indicate to the autonomous driving computersystem 144 the amount of change in the driving environment. Accordingly,based on one or more of these metrics, the autonomous driving computersystem 144 may determine whether a driving environment has changed.

Furthermore, the autonomous driving computer system 144 may leverage aprobability function that provides a probability value (or confidencevalue) that the driving environment has changed. The probabilityfunction may refer to one or more probability models to determinewhether a driving environment has changed. As input, the probabilityfunction may accept one, or a combination of, the foregoing deviationmetrics, such as the maximum deviation metric, the average deviationmetric, the average signed deviation metric, and so forth.

The autonomous driving computer system 144 may also store one or moreprobability models, such as a probability model for a highway, aprobability model for a parkway, and so forth, and the autonomousdriving computer system 144 may select which of the probability modelsto use in the probability function based on the location, or drivingenvironment, of the autonomous vehicle 104. In this manner, theprobability function may provide a more granular determination ofwhether the driving environment has changed (e.g., 83%) in contrast tothe determination of whether the driving environment has changed basedon comparing the deviation metric values with corresponding thresholds(e.g., yes, where the deviation metric is greater than the threshold;no, where the deviation metric is not greater than the threshold).

In addition to relying on metrics determined for individualtrajectories, the autonomous vehicle computer system 144 may determinemetrics for trajectories where the corresponding vehicles are within apredetermined distance, or distance threshold, to the autonomous vehicle104. The autonomous vehicle computer system 144 may also determinemetrics for consolidated trajectories. These approaches are describedwith reference to FIGS. 9-11 .

In one embodiment, the autonomous vehicle 104 may detect one or morevehicles travelling in front of the autonomous vehicle 104 and one ormore vehicles travelling behind the autonomous vehicle 104. FIG. 9illustrates an example where the autonomous vehicle 104 is in a drivingenvironment that has denser traffic than the driving environment shownin FIG. 6 . For simplicity, the detailed map 502 of FIG. 5 is used inFIG. 9 .

Although FIG. 9 illustrates that the autonomous vehicle 104 may detectvehicles 904-908 and vehicles 910-914 travelling in front and behind,respectively, the autonomous vehicle 104 may also detect vehiclestravelling in a different lane (e.g., any of vehicles 916-922) orvehicles travelling in a different direction, such as vehiclestravelling in an oncoming direction relative to the autonomous vehicle104.

Based on detecting and monitoring other vehicles, the autonomous drivingcomputer system 144 may determine corresponding raw trajectories (notshown). As discussed previously, the autonomous driving computer system144 may determine the raw trajectories based on the stored vehicle data116, such as a vehicle's position, direction, speed, and other suchvehicle data 116. These raw trajectories may be individually averaged toproduce smoothed trajectories. FIG. 10 illustrates an example 1002 wherethe raw trajectories for vehicles have been averaged to producecorresponding smoothed trajectories 1004-1016.

The example 1002 illustrates the lateral distances of the smoothedtrajectories 1004-1016 from the autonomous vehicle 104. For referencepurposes, a first smoothed trajectory 1004 may correspond to vehicle904, a second smoothed trajectory 1006 may correspond to vehicle 908, athird smoothed trajectory 1008 may correspond to vehicle 910, a fourthsmoothed trajectory 1012 may correspond to vehicle 906, a fifth smoothedtrajectory 1014 may correspond to vehicle 914, and a sixth smoothedtrajectory 1016 may correspond to vehicle 912. The smoothed trajectory1010 may correspond to the autonomous vehicle 104.

As with the examples of 8A and 8B, in the example of FIG. 10 , theautonomous driving computer system 144 may determine deviation metricvalues based on comparing the smoothed trajectories with the previouslystored detailed map information 114 (e.g., the portion of the detailedmap 502). In particular, the autonomous driving computer system 144 maydetermine the deviation metric values 1018-1024 based on comparing thesmoothed trajectories 1004-1016 with an expected trajectory 1026. In oneembodiment, the expected trajectory 1026 may be the centerline rail 518of the lane in which the autonomous driving computer system 144estimates that the autonomous vehicle 104 is traveling. In otherembodiments, the autonomous driving computer system 144 may use anexpected trajectory derived from empirical data of vehicles traveling inthe driving environment.

The autonomous vehicle 104 may determine which of the smoothedtrajectories 1004-1016 to use when determining deviation metric valuesbased on the distance between the vehicles corresponding to the smoothedtrajectories and the autonomous vehicle 104. With reference to FIG. 9 ,the autonomous vehicle 104 may be capable of detecting vehicles 904-914.However, as vehicle data (e.g., speed, direction, type, etc.) for avehicle may become less reliable as the distance between the vehicle andthe autonomous vehicle 104 increases, the autonomous driving computersystem 144 may employ various distance thresholds, such as distancethresholds 924-926, to determine which vehicle trajectories to use indetermining the deviation metric values 1018-1024.

The distance thresholds may be any measure of distance, such as meters,feet, yards, inches, centimeters and so forth. In one embodiment, afront distance threshold 924 and a rear distance threshold 926 may be150 feet. Although not shown, the autonomous vehicle 104 may also havelateral distance thresholds, such as a left lateral distance thresholdand a right lateral distance threshold. The autonomous vehicle 104 mayemploy any combination of distance thresholds to limit which vehicletrajectories to use in determining the deviation metric values.

As previously discussed, the autonomous vehicle 104 may determinefurther deviation metric values based on the deviation metric valuesdetermined from comparing smoothed trajectories with an expectedtrajectory. As discussed previously, the further deviation metrics mayinclude one or more maximum deviation metrics (e.g. a maximum deviationmetric for each individual trajectory or a maximum deviation metric forthe set of trajectories), one or more average deviation metrics, and oneor more average signed deviation metrics. These deviation metrics mayhave corresponding thresholds (i.e., a maximum deviation metricthreshold, an average deviation metric threshold, and an average signeddeviation metric threshold) that the autonomous driving computer system144 uses to determine whether the driving environment has changed.

Similarly, as discussed with reference to FIGS. 8A-8B, the autonomousdriving computer system 144 may also use a probability function and oneor more probability models to determine whether the driving environmenthas changed. In this regard, the autonomous driving computer system 144may select a probability model based on the driving environment orlocation of the autonomous vehicle 104. The autonomous driving computersystem 144 may then provide the probability function with the selectedprobability model, the maximum deviation metric, the average deviationmetric, and/or the average signed deviation metric. The result of theprobability function may indicate the probability that the drivingenvironment has changed relative to the detailed map information 114.

As noted above, in addition to, or as an alternative, the autonomousdriving computer system 144 may use consolidated trajectories fordetermining whether the driving environment has changed relative to thedetailed map information 114. In FIG. 11 , the autonomous drivingcomputer system 144 has determined a first consolidated trajectory 1104and a second consolidated trajectory 1106. The consolidated trajectories1104,1106 may represent an average of two or more of the smoothedtrajectories 1004-1016 or an average of two or more of the rawtrajectories for the vehicles detected by the autonomous drivingcomputer system 144.

More particularly, which trajectory, or trajectories, to include in aconsolidated trajectory may be based on each smoothed (or raw)trajectory's relationship to the trajectory of the autonomous vehicle104 or the relationship between each trajectory's corresponding vehicleand the autonomous vehicle 104. For example, trajectories that aredetermined to be to the left of the autonomous vehicle may beconsolidated into one consolidated trajectory and trajectories that aredetermined to be to the right of the autonomous vehicle may beconsolidated into another consolidated trajectory. Similar groupings maybe performed for trajectories determined to be in other lanes, fortrajectories determined to be from oncoming traffic, or other similargroupings.

For example, and with reference to FIG. 10 , the autonomous drivingcomputer system 144 has determined that smoothed trajectories 1004-1008are each to the left of the trajectory 1010 for the autonomous vehicle104. Similarly, the autonomous driving computer system 144 hasdetermined that smoothed trajectories 1012-1016 are each to the right ofthe trajectory 1010 for the autonomous vehicle 104. Accordingly, in oneembodiment, smoothed trajectories 1004-1008 may be consolidated into asingle, consolidated trajectory (i.e., consolidated trajectory 1104),and smoothed trajectories 1012-1016 may be consolidated into anothersingle, consolidated trajectory (i.e., consolidated trajectory 1106).

In another embodiment of consolidating trajectories, the smoothedtrajectories may be consolidated based on portions of the trajectoriesthat share overlapping portions. As there are times that vehicles travelin relatively straight lines, such as on a highway or the like, it wouldnot be unexpected for more than one vehicle to have similar, or nearlyidentical trajectories. For example, with reference to FIG. 9 , onemight expect that vehicle 904 and vehicle 910 to have similartrajectories. As another example, one might expect that vehicle 906 andvehicle 912 to have similar trajectories. Accordingly, there may beportions of the trajectories for vehicle 904 and vehicle 910 thatoverlap (or are within a degree of tolerance, e.g., three or fourcentimeters), and there may be portions of the trajectories for vehicle906 and vehicle 912 that overlap (or are also within a degree oftolerance, e.g., three or four centimeters). Based on detecting thatportions of these trajectories overlap, the autonomous driving computersystem 144 may consolidate these trajectories. Consolidatingtrajectories based on which portions overlap may include consolidatingthe complete trajectories, only the portions of those trajectories thatoverlap (e.g., two vehicles may have similar trajectories, but one ofthose vehicles may change lanes), or a combination of the foregoing.

Referring back to FIG. 11 , the autonomous driving computer system 144may then determine deviation metric values 1108-1110 by comparing theconsolidated trajectories with the expected trajectory 1026. Asdiscussed with reference to FIG. 10 , the expected trajectory 1026 maybe based on the centerline rail 518 of the lane in which the vehiclesare traveling, on empirical data from previously monitoring andobserving other vehicles, or a combination of the foregoing. Based onthe determined deviation metric values 1108-1110, the autonomous drivingcomputer system 144 may also determine one or more maximum deviationmetric values (e.g., a maximum deviation metric value for eachindividual trajectory 1108-1110 or a maximum deviation metric value forthe set of trajectories 1108-1110), one or more average deviation metricvalues, one or more average signed deviation metric values, theprobability value that the driving environment has changed, and soforth.

Furthermore, one of the challenges in determining whether the drivingenvironment has changed relative to the detailed map information 114based on consolidated trajectories (e.g., trajectories 1104-1106), isthat there may be extraneous errors introduced into the consolidatedtrajectories from the underlying smoothed or raw trajectories.Accordingly, the autonomous driving computer system 144 may determine aconsolidated trajectory quality value that indicates the quality of theconsolidated trajectory. Factors that may change the consolidatedtrajectory quality value may include the number of trajectories used inthe consolidation, the noisiness of the individual trajectories, theaccuracy of the individual trajectories, and other such factors.Increasing the number of individual trajectories used, using relativelyclean (e.g., less noisy) individual trajectories, and using relativelyaccurate trajectories may produce a greater consolidated trajectoryquality value (e.g., a value of 80 out of 100). In contrast, a fewernumber of individual trajectories used, noisiness in the individualtrajectories, and using relatively inaccurate trajectories may produce alower consolidated trajectory quality value (e.g., a value of 20 out of100). In one embodiment, the consolidated trajectory quality value maybe a multiplier that scales one or more of the determined metric valuesand/or probability value to account for the quality of the consolidatedtrajectory.

Accordingly, in addition to determining the one or more maximumdeviation metric values, the one or more average deviation metricvalues, the one or more average signed deviation metric values, and/orthe probability value based on the consolidated trajectories fordetermining whether the driving environment has changed relative to thedetailed map information 114, the autonomous driving computer system 144may also determine the consolidated trajectory quality value when one ormore trajectories are being consolidated.

FIG. 12 illustrates a first example of logic flow 1202 for determiningwhether a driving environment has changed based on determinedtrajectories for one or more detected vehicles according to aspects ofthis disclosure. Initially, the autonomous driving computer system 144may detect and track one or more vehicles in a driving environment(Block 1204). The autonomous driving computer system 144 may detect theone or more vehicles by using one or more of the sensors mounted on theautonomous vehicle 104, such as the laser sensors 302-304, radardetection units 306-312, the cameras 314-316, or a combination of thesesensors.

The autonomous driving computer system 144 may then store stateinformation about the detected vehicles as vehicle data 116. Aspreviously discussed, the state information about the detected vehiclesmay include vehicle speed, vehicle direction, the type of vehicle, thedistance between the autonomous vehicle 104 and the detected vehicle,and other such state information.

From the state information of the detected vehicles, the autonomousdriving computer system 144 may determine one or more raw trajectoriesfor corresponding detected vehicles (Block 1206). A raw trajectory maycomprise positional, speed, and direction information for acorresponding vehicle over a given time frame. The autonomous drivingcomputer system 144 may then determine an averaged, or smoothed,trajectory from the raw trajectory (Block 1208). In one embodiment, theautonomous driving computer system 144 may determine the smoothedtrajectory by averaging the positional information of the raw trajectoryover the time frame, in its entirety or a portion thereof.

The autonomous driving computer system 144 may then determine one ormore deviation metric values from the averaged one or more trajectories(Block 1210). As previously discussed, the deviation metric values maybe determined by comparing an averaged trajectory with an expectedtrajectory determined, or obtained from, the detailed map information114. In one embodiment, the expected trajectory may be a centerline of alane in which the autonomous driving computer system 144 determines thata detected vehicle is traveling.

The autonomous driving computer system 144 may then determine whether achange in the driving environment has occurred relative to the detailedmap information 114. For example, the autonomous driving computer system144 may compare one or more of the determined metric values to acorresponding metric threshold. Should one or more of the determinedmetric values exceed their corresponding threshold, the autonomousdriving computer system 144 may determine that the driving environmenthas changed.

Further still, the autonomous driving computer system 144 may leveragethe probability function to determine the probability that the drivingenvironment has changed relative to the detailed map information 114. Inone embodiment, the autonomous driving computer system 144 may select aprobability model corresponding to the driving environment of theautonomous vehicle 104 (Block 1212). The selected probability model mayalso correspond to the detailed map information 114. The autonomousdriving computer system 144 may then use one or more of the determinedmetrics and the selected probability model as inputs to the probabilityfunction. The probability function may then yield the probability thatthe driving environment has changed (Block 1214).

Although not shown in FIG. 12 , the autonomous driving computer system144 may take one or more actions based on the determined probabilitythat the driving environment has changed. In one embodiment, one or moreprobability thresholds may be established that correspond to the one ormore actions, and the autonomous driving computer system 144 may performthese actions when one or more of these probability thresholds areexceeded.

For example, a first probability threshold, such as 50%, may correspondwith an action to display a warning. Thus, when the probability that thedriving environment has changed equals or exceeds this first threshold,the autonomous driving computer system 144 may display a warning thatthe driving environment may have changed. In another example, a secondprobability threshold, such as 75%, may correspond with an action forthe autonomous driving computer system 144 to update the detailed mapinformation, such as by communication with the map provider server 142.Thus, when the probability that the driving environment has changedequals or exceeds this second threshold, the autonomous driving computersystem 144 may request updated detailed map information from the mapprovider server 142. Further still, where this second probabilitythreshold is met or exceeded, the autonomous driving computer system 144may also display a warning regarding the determined change in thedriving environment (e.g., the first probability threshold having beenexceeded).

Referring to FIG. 13 is another example of logic flow 1302 fordetermining whether a driving environment has changed based ondetermined trajectories for one or more vehicles. In FIG. 13 , theautonomous driving computer system 144 may determine whether the drivingenvironment has changed relative to the detailed map information 114based on one or more consolidated trajectories.

Similar to the logic flow 1202 of FIG. 12 , the logic flow 1302 of FIG.13 may include detecting one or more vehicles in a driving environment(Block 1304), determining one or more raw trajectories from the detectedvehicles (Block 1306), and then determining one or more averagedtrajectories from the determined raw trajectories (also Block 1306).

Thereafter, the autonomous driving computer system 144 may consolidateone or more of the trajectories into one or more consolidatedtrajectories (Block 1308). As discussed previously, a consolidatedtrajectory may represent an average of one or more of the determinedtrajectories (including raw trajectories, estimated trajectories, or acombination of the two).

More particularly, the autonomous driving computer system 144 mayconsolidate the determined trajectories based on one or moreconsolidation factors. These factors may include a trajectory'srelationship to the autonomous vehicle 104 or the trajectory of theautonomous vehicle, the quality of the trajectory (e.g., its accuracy,noisiness, duration, etc.), and other such factors.

After consolidating the one or more determined trajectories into one ormore consolidated trajectories, the autonomous driving computer system144 may then determine one or more deviation metric values (Block 1310).The autonomous driving computer system 144 may also determine one ormore consolidated trajectory quality values that indicate the quality ofone or more of the consolidated trajectories (Block 1312). As discussedpreviously, a consolidated trajectory quality value may be a multiplierthat scales one or more of the deviation metrics, the probability valuethat the driving environment has changed, or a combination thereof.

The autonomous driving computer system 144 may then determine whether achange in the driving environment has occurred relative to the detailedmap information 114. For example, the autonomous driving computer system144 may compare one or more of the determined metric values to acorresponding metric threshold. Should one or more of the determinedmetric values exceed their corresponding threshold, the autonomousdriving computer system 144 may determine that the driving environmenthas changed.

Further still, the autonomous driving computer system 144 may select aprobability model for use in determining whether the driving environmenthas change (Block 1314). The probability model may be associated withthe location of the autonomous vehicle 104, the driving environment, thedetailed map information 114, or a combination thereof. As discussedpreviously, the autonomous driving computer system 144 may store varyingprobability models for different driving environments.

Finally, the autonomous driving computer system 144 may then determinethe probability that the driving environment has changed (Block 1316).In one embodiment, the autonomous driving computer system 144 maydetermine whether the driving environment has changed by passing thedetermined deviation metric values, the consolidated trajectory qualityvalue, and/or the selected probability model to a probability function.The result of the probability function may indicate a probability thatthe driving environment has changed. As discussed previously, one ormore probability thresholds may be established in the autonomous drivingcomputer system 144 that correspond to actions, such as displaying awarning, requesting updating detailed map information, etc., and theautonomous driving computer system 144 may perform one or more of theseactions based on comparing the result of the probability function withone or more of these probability thresholds.

In this manner, the autonomous driving computer system 144 determineswhether a driving environment has changed relative to previously storeddetailed map information 114. As the driving environment may changedsince the detailed map information was last updated in the autonomousdriving computer system 144, the autonomous driving computer system 144may rely monitoring the behavior of surrounding vehicles to determinewhether a change has occurred. Further still, the autonomous drivingcomputer system 144 may monitor for consistent behavior changes by thesurrounding vehicles to determine whether the driving environment haschanged. In particular, the autonomous driving computer system 144 maydetermine one or more trajectories for the surrounding vehicles, andthen may compare these trajectories to an expected trajectory for ahypothetical vehicle. Where there are consistent differences between thedetermined trajectories and the expected trajectory, indicated byvarious types of deviation metrics, the autonomous driving computersystem 144 may determine whether the driving environment has changed.

Although aspects of this disclosure have been described with referenceto particular embodiments, it is to be understood that these embodimentsare merely illustrative of the principles and applications of thepresent disclosure. It is therefore to be understood that numerousmodifications may be made to the illustrative embodiments and that otherarrangements may be devised without departing from the spirit and scopeof this disclosure as defined by the appended claims. Furthermore, whilecertain operations and functions are shown in a specific order, they maybe performed in a different order unless it is expressly statedotherwise.

1. A method comprising: receiving, by one or more processors of a firstvehicle, sensor information including information about vehicles in adriving environment of the first vehicle; determining, by the one ormore processors, a plurality of trajectories corresponding to thevehicles; consolidating, by the one or more processors, the plurality oftrajectories; determining, by the one or more processors, a consolidatedtrajectory quality value for the consolidated trajectory, theconsolidated trajectory quality value representing a quality of theconsolidated trajectory; determining, by the one or more processors, aprobability value that the driving environment has changed based on theconsolidated trajectory quality value; and performing, by the one ormore processors, one or more actions when the probability value has metor exceeded a probability threshold.
 2. The method of claim 1, whereinthe consolidated trajectory quality value varies based on a number oftrajectories, a noisiness of each of the plurality of trajectories, oran accuracy of each of the plurality of trajectories.
 3. The method ofclaim 1, wherein the consolidating the plurality of trajectoriesincludes determining raw trajectories from the vehicles and determiningaveraged trajectories from the determined raw trajectories.
 4. Themethod of claim 3, wherein the consolidated trajectory represents anaverage of the determined raw trajectories.
 5. The method of claim 3,wherein the consolidated trajectory represents an average of thedetermined averaged trajectories.
 6. The method of claim 3, wherein theconsolidated trajectory represents an average of the determined rawtrajectories and the determined averaged trajectories.
 7. The method ofclaim 1, wherein consolidating the plurality of trajectories of thevehicles is based on at least one consolidation factor.
 8. The method ofclaim 7, wherein the at least one consolidation factor is a relationshipof each of the plurality of trajectories to a trajectory of the firstvehicle.
 9. The method of claim 7, wherein the at least oneconsolidation factor is a quality of each of the plurality oftrajectories.
 10. The method of claim 1, wherein the one or more actionsis displaying a warning on a display of the first vehicle.
 11. Themethod of claim 1, wherein the one or more actions is requesting updatedmap information from a remote computing device.
 12. The method of claim1, wherein the probability value that the driving environment haschanged is determined by a probability function that refers to aprobability model.
 13. The method of claim 12, wherein the probabilitymodel is selected from a plurality of probability models based on a typeof the driving environment.
 14. The method of claim 13, wherein at leastone of the plurality of probability models is a highway-type probabilitymodel.
 15. A system comprising one or more processors configured to:receive sensor information including information about vehicles in adriving environment of a first vehicle; determine a plurality oftrajectories corresponding to the vehicles; consolidate the plurality oftrajectories; determine a consolidated trajectory quality value for theconsolidated trajectory, the consolidated trajectory quality valuerepresenting a quality of the consolidated trajectory; determine aprobability value that the driving environment has changed based on theconsolidated trajectory quality value; and perform one or more actionswhen the probability value has met or exceeded a probability threshold.16. The system of claim 15, wherein the consolidated trajectory qualityvalue varies based on a number of trajectories used in theconsolidation, a noisiness of each of the plurality of trajectories, oran accuracy of each of the plurality of trajectories.
 17. The system ofclaim 15, wherein the consolidation includes determining rawtrajectories from the vehicles and determining averaged trajectoriesfrom the determined raw trajectories.
 18. The system of claim 17,wherein the consolidated trajectory represents an average of thedetermined raw trajectories.
 19. The system of claim 17, wherein theconsolidated trajectory represents an average of the determined averagedtrajectories.
 20. The system of claim 17, wherein the consolidatedtrajectory represents an average of the determined raw trajectories andthe determined averaged trajectories.